Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kvserver: Add a metric for in-progress snapshots #99843

Merged
merged 1 commit into from
Mar 30, 2023

Conversation

andrewbaptist
Copy link
Collaborator

@andrewbaptist andrewbaptist commented Mar 28, 2023

Fixes: #98242

Knowing how many delegate snapshot requests are currently in-progress will be useful for detecting problems. This change adds a metric for this. It also updates the names of the previous stats to have the prefix range.snapshots vs range.snapshot to be consistent with other stats.

Epic: none

Release note: None

@andrewbaptist andrewbaptist requested a review from a team March 28, 2023 18:54
@andrewbaptist andrewbaptist requested a review from a team as a code owner March 28, 2023 18:54
@blathers-crl
Copy link

blathers-crl bot commented Mar 28, 2023

It looks like your PR touches production code but doesn't add or edit any test code. Did you consider adding tests to your PR?

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@andrewbaptist andrewbaptist force-pushed the 23.03.28-delegate-metrics branch from cb9433a to 842d2b8 Compare March 28, 2023 18:55
@andrewbaptist andrewbaptist requested a review from kvoli March 28, 2023 18:56
@andrewbaptist andrewbaptist force-pushed the 23.03.28-delegate-metrics branch from 842d2b8 to e0a1366 Compare March 28, 2023 19:19
Copy link
Collaborator

@kvoli kvoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: with release note for renaming failure/success metrics.

Reviewed 2 of 3 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @andrewbaptist)


-- commits line 9 at r2:
range.snapshots.delegate.failures and range.snapshot.delegate.successes had a release note when they were added (4a43cf6). You should add another release note here mentioning the rename, for docs.


pkg/kv/kvserver/replica_command.go line 2859 at r2 (raw file):

		)
		if !selfDelegate {
			r.store.Metrics().DelegateSnapshotInProgress.Dec(1)

nit: Did you consider moving this up above in the same if block as the inc and deferring the dec?

@andrewbaptist andrewbaptist force-pushed the 23.03.28-delegate-metrics branch from e0a1366 to bea9903 Compare March 28, 2023 19:55
Copy link
Collaborator Author

@andrewbaptist andrewbaptist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR - I added the release note.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @kvoli)


-- commits line 9 at r2:

Previously, kvoli (Austen) wrote…

range.snapshots.delegate.failures and range.snapshot.delegate.successes had a release note when they were added (4a43cf6). You should add another release note here mentioning the rename, for docs.

Done


pkg/kv/kvserver/replica_command.go line 2859 at r2 (raw file):

Previously, kvoli (Austen) wrote…

nit: Did you consider moving this up above in the same if block as the inc and deferring the dec?

I did consider this, but then it complains about a defer within a loop. I'm not sure which is better actually but I left this for now.

Copy link
Collaborator

@kvoli kvoli left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @andrewbaptist)

@andrewbaptist andrewbaptist force-pushed the 23.03.28-delegate-metrics branch from bea9903 to b2831cd Compare March 29, 2023 19:10
Fixes: cockroachdb#98242

Knowing how many delegate snapshot requests are currently in-progress
will be useful for detecting problems. This change adds a metric for
this. It also updates the names of the previous stats to have the prefix
`range.snapshots` vs `range.snapshot` to be consistent with other stats.

Epic: none

Release note (ops change): Adds a new stat
range.snapshots.delegate.in-progress and renames two existing stats.
They were never part of a release, so better to rename them before
23.1.0 is cut.
range.snapshot.delegate.successes -> range.snapshots.delegate.successes
range.snapshot.delegate.failures -> range.snapshots.delegate.failures
@andrewbaptist andrewbaptist force-pushed the 23.03.28-delegate-metrics branch from b2831cd to 36ae825 Compare March 29, 2023 22:24
@andrewbaptist
Copy link
Collaborator Author

bors r=kvoli

@craig
Copy link
Contributor

craig bot commented Mar 30, 2023

Build failed:

@knz
Copy link
Contributor

knz commented Mar 30, 2023

flake here #100119

bors r=kvoli

@craig
Copy link
Contributor

craig bot commented Mar 30, 2023

Build succeeded:

@craig craig bot merged commit 476cd85 into cockroachdb:master Mar 30, 2023
@andrewbaptist andrewbaptist added the backport-23.1.x Flags PRs that need to be backported to 23.1 label Mar 30, 2023
@knz
Copy link
Contributor

knz commented Apr 2, 2023

Adding the backport label after the PR has merged does nothing. You need to instruct blathers, as follows:

blathers backport 23.1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-23.1.x Flags PRs that need to be backported to 23.1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add metrics for in-progress delegations
4 participants